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A new sparse signal recovery algorithm for multiple-measurement vec- 
tors (MMV) problem is proposed in this paper. The sparse representation 
is iteratively drawn based on the idea of zero-point attracting projection 
^, ! (ZAP). In each iteration, the solution is first updated along the negative 

gradient direction of sparse constraint, and then projected to the solution 

>■ 

00 ■ space to satisfy the under-determined equation. A variable step size scheme 

^. 
•/^ . is adopted further to accelerate the convergence as well as to improve the 

fT^ ' recovery accuracy. Numerical simulations demonstrate that the performance 

o' 

CN ' of the proposed algorithm exceeds the references in various aspects, as well 

as when applied to the Modulated Wideband Converter, where recovering 

^ I MMV problem is crucial to its performance. 

H ■ 
_^_' Keywords: sparse recovery, multiple-measurement vectors, zero-point 

attracting projection, the Modulated Wideband Converter 



* Correspondence should be addressed to Yuantao Gu at Department of Electronic 
Engineering, Tsinghua University, Beijing 100084, China. Email: gyt@tsinghua.edu.cn. 



Preprint submitted to Elsevier February 25, 2013 



1. Introduction 

Sparse recovery is one of the essential issues in many fields of signal pro- 
cessing, including compressive sampling (CS) [l|,l2|, which is a novel sampling 
theory. In some applications such as magnetoencephalography (MEG) |3|, 
equalization of sparse communication channels J4J and analog-to-digital con- 
version |5|, the measurements and sparse unknowns are both in high dimen- 
sion, which results in the problem of multiple measurement vectors (MMV). 
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In the Modulated Wideband Converter (MWC) [51], the locations of narrow 
band signals in wide spectrum range are detected by solving MMV problem, 
which is of great significance to the performance of the conversion system. 

Several algorithms are proposed to derive the sparse solution to MMV 
problem. OMP algorithm for MMV [61] finds a sparse solution by sequentially 
building up a small subset of column vectors selected from A to represent 
Y. FOCal Underdetermined System Solver (FOCUSS) methods |3| compute 
the constraint solution in an iterative procedure using the standard method 
of Lagrange multipliers. ReMBo 7| reduces MMV to SMV by multiplying 
a random column vector drawn from an absolutely continuous distribution. 
Recently, a robust algorithm for joint-sparse recovery named Joint /2.0 Ap- 
proximation algorithm (JLZA) 8| is proposed to calculated the solution in a 
fixed point iteration with an approximation of ^2,0 norm. Different algorithms 
which aim at solving the problem of MMV are surveyed and compared in 9|. 

This work extends a recently proposed zero-point attracting projection 
(ZAP) algorithm 10| to the MMV scenario. Derived from the adaptive fil- 
tering framework, ZAP defines the cost function as the sparsity and updates 
the sparse representation in the solution space. Specifically, starting from the 



least squares solution, it is modified in a fixed step-size along the negative 
gradient direction of sparse constraint towards zero point, and then projected 
back to the solution space again. This operation is executed iteratively until 
the stop conditions are satisfied. In this paper, an approximate ^2,0 norm is 
adopted in ZAP and the recursive solution to MMV is derived. In addition, 
the step-size of zero-point attraction is varied dynamically to accelerate the 
convergence. The rest of the present paper is organized as follows. Section [2] 
formulates the MMV problem and the approximate /2,o norm. Section [3] 
introduces ZAP algorithm for MMV, and further discussions and improve- 
ments are given in Section HI Section |5] shows the simulation results and this 
paper is concluded in Section |6l 

2. Problem Formulation 

The problem of MMV is formulated as Y = AX, where Y G M^^""^ 
is measurement matrix, A G R*^^^ is sensing matrix, and X G R^^^ is 
unknown signal assumed jointly ii"-sparse, i.e. it has K nonzero rows at 
most. It is shown that the following problem 

X = argmin ||r(X)||o, subject to Y = AX (1) 

finds the unique i^-sparse solution, where r(X) returns a column vector 
whose zth item is the norm of the ith row of X, and || • ||o counts nonzero 
elements of the vector. 

According to the references 3|, |8|], we choose ^2,0 norm 

J(X) = ||r2(X)||o (2) 



to describe the sparsity of a matrix, where r2(X) denotes a column vector 
with I2 norm of the zth row of X as its ith item. The sparsity penalty can 
be further approximated by a continuous function 

N 

J(X) = ^F„(||x,||2), (3) 

where Xj denotes the ith row of X, and the function Fa{-) is defined as 

{2a\w\ — c?w^ \w\ < 7^: 
1 elsewhere. 

It is readily recognized that Fa{w) converges to /q norm as a approaches to 
infinity. The derivative of -Fq(-) is a segmented continuous linear function 

{2a ■ sgn(w) — 2a^w \w\ < ^\ 
elsewhere, 

where sgn(-) denotes the sign function. 

3. Zero-point Attracting Projection for MMV 

In this section we extend ZAP to the MMV scenario. Further discussions 
and improvements of this algorithm will be shown in section |H 

ZAP for MMV derives the sparse solution in an iterative procedure. In the 
nth iteration, the solution is first updated along negative gradient direction 
of sparse penalty ([3]), 

X(n) = X(n-l)-fi:- VJ(X(r2-l)), (6) 

where X(n) denotes the temporary solution, k > denotes the step size, and 
VJ(-) is a gradient matrix with partial derivative 



as its {i,j)th element, where Xij stands for the {i,j)th entry of matrix X. 
Substituting (jl]) into (|3]), and together with (j5]), the {i,j)th element of the 
gradient matrix can be calculated 

*'^ dXi,j ||Xi||2 

Please notice that the I2 norm of each row ||xi||2 appears at denominator 
in ([8]) may cause unstable problem, because most rows in a jointly sparse 
matrix are zero. Thus a small positive factor 6 is added to each denominator. 
Consequently, the entire gradient matrix is 

VJ(X) = diag , . . . , II II . X. (9) 

\||Xi||2 + () W^Nh + Oj 

Then the solution is projected to the solution space to satisfy Y = AX 

X(r2) =X(n) + A^(Y- AX(r2)), (10) 

where A^ = A"^(AA^)^^ denotes the pseudo- inverse matrix of A. 

Now we have introduced the basic procedure of ZAP for MMV problem. 
Compared with solving the MMV problem column by column independently, 
ZAPMMV utilizes the information of joint sparsity by means of iQ. The 
updating step of each element is renormalized according to the magnitude of 
the corresponding row, which greatly improves the recovery performance. A 
detailed discussion about the effect of the parameters and a fast convergence 
version of the proposed algorithm is given in the following section. 

4. Discussions and Improvements 

So far, the parameters involved in the ZAP for MMV problem are Iq 
norm approximation factor a and step size k, and these parameters should 
be selected carefully. 



The choice of a: The value of a has a great impact on the performances 
of the proposed algorithm. By (|5]) we know that large a pulls small values 
to zero quickly but the range of its influence is small, while small one has 
the opposite effect. Furthermore, according to @, large a leads to a bet- 
ter approximation of /q norm, but simultaneously, produces too many local 
minima in the cost function [8|, where the solution may result in converging 
to some local minimum. In practice, we suggest 1/a to be several times the 
expectation of I2 norm of nonzero rows in X. 

The choice of n: As the step size in gradient descent iterations, the 
parameter k suggests a tradeoff between the speed of convergence and the 
accuracy of the solution. Big k indicates faster convergence but less accuracy, 
while small k yields more accuracy solution but causes slower convergence. 

In order to improve the performances of the proposed algorithm, the idea 
of variable step size is taken into consideration. The control scheme is rather 
direct: k is initialized to be a large value, and reduced by a factor r] as long 
as the iteration is convergent. The reduction is repeated several times until k 
is sufficiently small, which means that the recovery error reaches a low level. 

The criterion of convergence: It is significantly important to select 
an appropriate criterion of convergence to make full use of variable step size. 
Considering the iteration is performed along the negative gradient direction of 
the sparsity penalty ([3]), it is obvious that the algorithm reaches steady-state 
when the penalty starts increasing. To reduce the complexity of evaluation 
and improve its stability, the criterion is checked every Q iterations. 

The initial value: Follow the ZAP algorithm for SMV, least squares 
solution is selected as the initial value, X(0) = A'I'Y. 



The stop conditions: The proposed algorithm stops (a) when the 
number of reductions of n reaches a fixed number D or (b) when the number 
of iterations reaches the bound T. 

5. Simulations 

In this section three experiments are designed to demonstrate the per- 
formance of the proposed algorithm in various aspects, such as recovery 
probability versus sparsity and robustness to measurement noise, as well 
as recovery performance in the Modulated Wideband Converter J5|. 

In the first two simulations, four existing algorithms including OMPSMV, 
OMPMMV [61, ReMBo [7], and JLZA [81, are simulated in the same scenar- 
ios for references. OMPSMV denotes the algorithm that X is recovered col- 
umn by column as SMV problem using orthogonal matching pursuit (OMP). 
Gaussian sensing matrix is adopted in all experiments. The locations of 
nonzero rows in the jointly sparse matrix are uniformly chosen at random 
among all possible choices, and the values of nonzero entries are i.i.d. stan- 
dard normal distribution. We define exact recovery when the relative error 
||X — X||f/||X||f is smaller than 10~^, where || ■ ||f denotes the Frobenius- 
norm. The experiments are conducted for 1000 trials to calculate the prob- 
ability of exact recovery or the average relative error. 

In ZAPMMV, we set a = 1, k = 0.1, t] = 0.1, 6 = 10-^ Q = 11, 
D = A, and T = 500. In the reference algorithms, the parameters are 
selected as suggested to yield the best performance. In ReMBo algorithm, 
the components of a G M"^ are drawn from standard normal distribution, and 
the maximum number of iterations allowed to recover X using different a is 
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Figure 1: The probability of exact recovery versus sparsity K. 
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The first experiment tests the recovery probability with respect to the 
sparsity. We set A^ = 200, M = 50, L = 10, and the sparsity K varies from 
2 to 50. Fig. [1] shows the result in this scenario. As can be seen, the recovery 
performance of ZAPMMV far exceeds those of the other algorithms. While 
all the other algorithms fail when K is larger than 20, the proposed algorithm 
guarantees exact recovery until K reaches 24. 

The second experiment studies the recovery performance against measure- 
ment noise. The measurements are supposed to be contaminated by noise 
as Y = AX + V, where V denotes the zero mean additive white Gaussian 
noise. Fig. |2] demonstrates the performance of different algorithms in noisy 
environment, where the measurement signal-to-noise ratio (SNR) varies from 
lOdB to 50dB. As can be seen from the figure, JLZA and ZAPMMV enjoy 
the best recovery performance in the presence of noise. 



N=200,M=50,K=10,L=10 



OMPSMV 

OMPMMV 

ReMBo-OMP 

JLZA 

ZAPMMV 




Figure 2: The recovery raean squared deviation (MSD) versus SNR. 



In the final experiment, OMPMMV and ZAPMMV are applied to solve 
the MMV problem in the MWC respectively, to compare their recovery per- 
formance. In our simulations, 30 channels with IMHz ADCs are utilized to 
sample a sparse signal at Nyquist rate IGHz, and each sampling sequence 
contains 999 pulses in one period. The original sparse signal contaminated 
by Gaussian white noise contains 3 narrow band components (0.5MHz of 
each and their locations are unknown). The waveform and the power spec- 
trum density (PSD) of the input signal with in-band SNR 14dB is depicted 
in Fig. 131 Fig. H] demonstrates the probability of successfully detecting the 
locations of all narrow band components, two narrow band components, and 
only one components, respectively, where the in-band SNR of original signal 
varies from 2dB to 20dB. The experiment is conducted for 5000 trials. It can 
be readily accepted that compared with the reference method, ZAPMMV 
can recover sparse signal with higher probability, as verified its performance 
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Figure 3: (a) waveform and (b, c) power spectrum density (PSD) of original signal with 
in-band SNR 14dB, where (c) is the detail of (b). The x-axis of (a) and (b, c) denotes 
time in second and frequency in GHz, respectively. 

against signal noise. 

6. Conclusion 

In this paper, we extend the zero-point attracting projection algorithm to 
solve the multiple-measurement vectors problem. An approximate ^2,0 norm 
is adopted as the sparsity penalty to force the recursion solution converging to 
the sparsest representation. To balance the convergence speed and recovery 
accuracy, a step size control scheme is introduced. The choices of parameters 
and related criterions are discussed briefly. Several experiments demonstrate 
that the proposed algorithm outperforms most available algorithms in re- 
spect of sparsity as well as measurement noise, and in the application of the 
Modulated Wideband Converter. 
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Figure 4: The recovery probability of OMPMMV and ZAPMMV in MWC simulation, 
where OMPMMV(z) and ZAPMMV(i) denote that i components are successfully detected. 
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